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(54) Image signal encoding apparatus 

(57) An interf rame difference Pij and a motion vector 
Vhv are supplied to a camera shake detecting circuit 
( 1 2) from a motion vector detecting circuit (6). The cam- 
era shake detecting circuit (1 2) integrates the interf rame 
difference and forms an evaluation value IPhv. When 
none of the blocks of one picture plane satisfy (Vhv < 
TH1 and IPhv > TH2) for threshold values TH1 and 
TH2, the camera shake detecting circuit (12) detects a 
camera shake. The motion vectors of the blocks are dis- 
tributed into macroblocks and converted into a motion 
vector of each macroblock. A camera shake motion vec- 
tor of the whole picture plane is formed from the motion 
vector of each macroblock. A correction signal is formed 
from the detected camera shake motion vector. A cam- 
era shake correcting process is executed using the cor- 
rection signal. 



CO 



- o a 



-i 









SSI 




PROCE 






8 


o 



Printed by Jouve. 75001 PARIS (FR) 



BNSDOCID: <EP 0762774A2_I_> 



EP 0 762 774 A2 



Description 

BACKGROUND OF THE INVENTION 

FIELD OF THE INVENTION 

The invention relates to an image signal processing 
apparatus for compressing image data such as a pho- 
tographing output of a video camera of a handy type or 
the like, an image signal recording apparatus for record- 
ing compression image data to a recording medium, and 
an image signal reproducing apparatus for reproducing 
the compression image data from the recording medium 
and, more particularly, to a correction of camera shake 
of a video camera. 

DESCRIPTION OF THE PRIOR ART 

When photographing by using a video camera of a 
handy type, there is a problem such that a reproduction 
picture plane is shaken due to camera shake. To solve 
the problem, there is considered a method whereby a 
motion vector is detected and image data stored in an 
image memory is corrected on the basis of the motion 
vector. The detection of the motion vector is performed 
by, for example, block matching. That is, the picture 
plane is divided into a number of areas (referred to as 
blocks), an absolute value of a frame difference be- 
tween a representative point of a previous frame which 
is located at the center of each block and pixel data in 
the block of the present frame is calculated and the ab- 
solute value of the frame difference is integrated with 
respect to one picture plane, thereby detecting the mo- 
tion vector of the whole picture plane from the position 
of the minimum value of integration frame difference da- 
ta obtained. The detected motion vector is converted in- 
to a correction signal and a correction to move an orig- 
inal image is performed by the correction signal. 

For example, in Fig. 1 A, a picture frame La shown 
by a broken line is a picked-up image and its position is 
corrected to a picture frame Lb shown by a broken line 
by a camera shake correction. In the image in the picture 
frame Lb after completion of the correction, since there 
is no pickup image in the area shown by the hatched 
portion, drop-out of images occurs. As one method of 
solving the problem, there is a method of enlarging the 
picture frame to some degree as shown by Lc in Fig. 1 B. 
Thus, it is possible to prevent the drop-out of images. 

However, the enlargement of the image makes the 
reading speed of the image memory slower than the 
writing speed of the memory and is performed by a proc- 
ess of interpolating the missing pixel data. Therefore, 
resolution of the enlarged image deteriorates as com- 
pared with that of the original image. Thus ; there is a 
problem that picture quality after completion of the cam- 
era shake correction is not good. 

Other problems in the conventional camera shake 
correction will now be described hereinbelow. There is 



a problem that motion of a large area in the picture plane 
is erroneously discriminated as camera shake. When 
using a method of dividing one picture plane into (4x4 
= 1 6) macroblocks and detecting a motion vector in each 
s macroblock, for example, as shown in Fig. 2A, when a 
person moves in the direction from the right to the left 
in the picture plane, the motion vector shown by an ar- 
row is detected in each block. The motion vectors in the 
other blocks in which no person (edge) is included are 
^0 detected as 0. 

Hitherto, the motion vector of each macroblock is 
discriminated by using a decision by the majority and 
the motion vectors as major vectors are used as motion 
vectors for the camera shake correction. As in the con- 
's ventional apparatus mentioned above, however, when 
an object having a large area exists at the center of the 
picture plane and the object moves, there is a possibility 
such that the movement of the object is erroneously dis- 
criminated as camera shake. It is, consequently, neces- 
20 sary to avoid such an error discrimination. 

Further, there is also a case where a plurality of mo- 
tion vectors exist as motion vectors of the macroblocks. 
For example, as shown in Fig. 2B, the motion vectors 
corresponding to the motion which occurred due to the 
25 camera shake and motion vectors which occurred due 
to the motion of the object in the picture plane are syn- 
thesized, so that a plurality of motion vectors are detect- 
ed. Like an example of Fig. 2B, there is a problem such 
that in a state in which there is a motion of the object 
30 having a large area, when the majority decision is per- 
formed as in the conventional apparatus, the motion 
vector due to camera shake cannot be correctly detect- 
ed. 

Further, in an image signal recording apparatus 
os such as a VTR of a video camera integrated type, it is 
standard practice to record a signal which was subject- 
ed to camera shake correction as an image signal to be 
recorded. Thus, however, the photographed image itself 
cannot be reproduced and, when the camera shake is 
40 not preferably corrected, there is a problem such that 
the correction is impossible on the reproducing side. 

As mentioned above, the conventional method of 
detecting the motion vector and correcting camera 
shake on the basis of the detected motion vectors has 
4 5 various problems. Further, in addition to the above-men- 
tioned problems, there is a problem that hardware to de- 
tect the motion vectors is necessary and the scale of the 
hardware for the camera shake correction increases. 

50 OBJECT AND SUMMARY OF THE INVENTION 

The invention intends to reduce the scale of hard- 
ware. Ordinarily, when digital image data is recorded to 
a recording medium (a magnetic tape, an optical disc, 
55 or the like), a motion compensation predictive encoding 
has been known as a highly-efficient encoding to com- 
press the amount of recording data. According to such 
an encoding, a difference between an input image signal 
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and a prediction image signal is detected, the difference 
is encoded, further, in the predictive encoding for varia- 
ble length encoding, when forming a predictive signal, 
a motion compensation by a detected motion vector is 
performed, thereby reducing the difference value. Since 5 
the motion vector is used in order to correct the camera 
shake of the video camera, the motion vector detected 
for motion compensation predictive encoding is used as 
a motion vector, so that hardware to detect the motion 
vector can be used in common. 10 

It is, therefore, an object of the invention to provide 
an image signal processing apparatus and a recording/ 
reproducing apparatus, in which a motion vector detect- 
ed for motion compensation predictive encoding can al- 
so be used in common to correct camera shake. is 

According to the invention, there is provided an im- 
age signal processing apparatus having functions for di- 
viding image data of one picture plane into a plurality of 
blocks, detecting a motion vector corresponding to a po- 
sition of the block which most coincides from the image 20 
data of one or a few frames before with respect to each 
block, and compressing the image data by using the mo- 
tion vector, comprising: a unit for detecting the motion 
vector of each block; a unit for detecting a camera shake 
motion vector from the motion vector of each block: a 25 
unit for correcting an input image signal on the basis of 
the camera shake motion vector; a unit for obtaining the 
motion vector in which the motion vector of each block 
has been corrected by the camera shake motion vector; 
a unit for performing a motion compensation by the cor- 30 
rected motion vector: and a unit for performing a com- 
pressing process by the image data which was camera 
shake corrected and the image data which was motion 
compensated. 

According to the invention, there is provided an im- 35 
age signal recording apparatus having functions for di- 
viding image data of one picture plane into a plurality of 
blocks, detecting a motion vector corresponding to a po- 
sition of the block which most coincides from image data 
of one or a few frames before with respect to each block, *o 
and compressing the image data by using the motion 
vector, comprising: a unit for detecting a motion vector 
of each block: a unit for detecting a camera shake mo- 
tion vector from the motion vector of each block: a unit 
for performing a motion compensation by the motion ^5 
vector: a unit for performing a compressing process by 
input image data and the image data which was motion 
compensated; and a unit for recording the camera 
shake motion vector together with the compressed im- 
age data. so 

Further, according to the invention, there is provid- 
ed an image signal reproducing apparatus for dividing 
image data of one picture plane into a plurality of blocks, 
detecting a motion vector corresponding to a position of 
the block which most coincides from the image data of ss 
one or a few frames before with respect to each block, 
compressing the image data by using the motion vector, 
detecting a camera shake motion vector from the motion 
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vector, performing a motion compensation by the motion 
vector, executing a compressing process by input image 
data and the image data which was motion compensat- 
ed, and reproducing a recording medium in which the 
camera shake motion vector has been recorded togeth- 
er with the compressed image signal, comprising: a de- 
coding unit for decoding the compressed image data; a 
unit for separating the camera shake motion vector from 
a reproduction signal; and a unit to which a decoded im- 
age signal from the decoding unit is supplied and which 
executes a camera shake correction by the camera 
shake motion vector. 

Like a motion compensation predictive encoding, in 
order to compress the recording data, a motion vector 
is detected. The motion vector is detected on a block 
unit basis. By using the motion vector and a change 
amount of the image data, the camera shake motion 
vector is detected. The image data which was corrected 
by the camera shake motion vector is recorded. Not only 
the camera shake corrected image data is recorded but 
also the camera shake motion vector or the correction 
signal formed by such a vector is recorded to the record- 
ing medium together with the compression image data, 
and the camera shake correction is executed upon re- 
production. 

The above and other objects and features of the 
present invention will become apparent from the follow- 
ing detailed description and the appended claims with 
reference to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figs. 1A and 1B are schematic diagrams for ex- 
plaining a conventional camera shake correcting 
process; 

Figs. 2A and 2B are further schematic diagrams for 
explaining the conventional camera shake correct- 
ing process; 

Fig. 3 is a block diagram of an embodiment of the 
invention; 

Fig. 4 is a block diagram showing constructions of 
a recording system and a reproducing system in the 
embodiment of the invention; 

Fig. 5 is a block diagram of an ADRC encoder in the 
embodiment of the invention: 
Fig. 6 is a block diagram of an ADRC decoder; 
Fig. 7 is a schematic diagram showing an ADRC 
encoding process; 

Fig. 8 is a schematic diagram for schematically ex- 
plaining a motion vector detection; 
Fig. 9 is a block diagram of an example of a motion 
vector detecting circuit; 

Fig. 10 is a schematic diagram for schematically ex- 
plaining a camera shake motion vector detection; 
Fig. 1 1 a block diagram of an example of a camera 
shake correcting circuit and a camera shake detect- 
ing circuit; 

Fig. 1 2 is a schematic diagram which is used in an 
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explanation of a detecting process of a camera 
shake motion vector; 

Fig. 1 3 is a further schematic diagram which is used 
in an explanation of the detecting process of the 
camera shake motion vector; 5 
Fig. 1 4 is a flowchart which is used in an explanation 
of the detecting process of the camera shake mo- 
tion vector; 

Fig. 1 5 is a schematic diagram showing an example 

of a process for dividing a picture plane into mac- 10 

roblocks: 

Fig. 1 6 is a flowchart showing the forming operation 
of an integrated vector in an embodiment of the in- 
vention; 

Figs. 1 7A and 1 7B are schematic diagrams showing is 
the camera shake correcting operation in the em- 
bodiment of the invention; 

Figs. 18Aand 18Bare schematic diagrams illustrat- 
ing the correcting operation of the motion vector in 
the embodiment of the invention; 20 
Fig. 19 is a block diagram of another embodiment 
of the invention; 

Fig. 20 is a block diagram of a recording system of 
still another embodiment of the invention; 
Fig. 21 is a block diagram of a reproducing system 2s 
for the Fig. 20 embodiment of the invention; 
Fig. 22 is a schematic diagram which is used for a 
schematic explanation of a background memory 
which is used in yet another embodiment of the in- 
vention; 30 
Fig. 23 is a block diagram of an embodiment of the 
invention using a background memory; and 
Fig. 24 is a block diagram showing a construction 
of an example of the motion vector detecting circuit 
for use in the Fig. 23 embodiment of the invention. 35 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

An embodiment of the invention will now be de- *o 
scribed hereinbelow with reference to the drawings. As 
an embodiment, for example, a video camera of a VTR 
integrated type is used, a photographed image signal is 
converted to a digital signal and is compressed by a mo- 
tion compensation predictive encoding and is recorded 
to a recording medium such a magnetic tape or the like. 

In Fig. 3, reference numeral 1 denotes a CCD as an 
image pickup device. An image pickup output of the 
CCD 1 is supplied to a camera signal processing circuit 
2 and is converted into a video signal. The video signal so 
is converted into a digital video signal by an A/D con- 
verter 3. The video signal is A/D converted by a sam- 
pling frequency of, for example, 1 3.5 MHz. Although the 
invention can be also applied to a component signal 
constructed by a luminance signal and two color differ- 55 
ence signals or a composite signal in which a luminance 
signal and a carrier chrominance signal have been su- 
perimposed, in order to simplify the explanation, only a 
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process of the luminance signal will now be described 
in the following description. 

The digital video signal from the A/D converter 3 is 
supplied through a delay circuit 14 to a camera shake 
correcting circuit 4 to which the invention is applied. As 
will be described hereinafter, the camera shake correct- 
ing circuit 4 is a circuit for correcting a camera shake 
occurring at the time of photographing. A camera shake 
detecting circuit 12 and a correction signal generating 
circuit 1 3 are provided for camera shake correction. The 
delay circuit 14 is provided for a phase adjustment to 
delay the digital video signal for the period of time which 
is required by the process to form the camera correction 
signal. An output signal of the cameral shake correcting 
circuit 4 is supplied to a subtracting circuit 5. An output 
signal of the A/D converter 3 is supplied to a motion vec- 
tor detecting circuit 6. 

A prediction signal from a motion compensating cir- 
cuit 7 is supplied to the subtracting circuit 5. Adifference 
between the actual video signal and the prediction sig- 
nal is generated for every pixel by the subtracting circuit 
5. The difference is supplied to an encoder 8 for ADRC 
encoding and an encoded output of the encoder 8 is ex- 
tracted. An output of the encoder 8 is supplied to an 
ADRC decoder (local decoder) 9, so that decoded data 
of a difference value is extracted as an output of the 
ADRC decoder 9. It is also possible to perform a refresh- 
ing process at the decoding side by periodically inserting 
an image signal of an intraframe encoding. 

The decoded value from the ADRC decoder 9 is 
supplied to an adding circuit 1 0 and is added to the pre- 
diction signal from the motion compensating circuit 7. 
The decoded signal obtained from the adding circuit 10 
is written into a frame memory 11 . An output of the frame 
memory 11 is supplied to the motion compensating cir- 
cuit 7. A motion vector Vhv from the motion vector de- 
tecting circuit 6 is supplied to the motion compensating 
circuit 7 through a motion vector correcting circuit 15, 
thereby performing a motion compensation. The predic- 
tion signal from the motion compensating circuit 7 is 
supplied to the adding circuit 10 and is also supplied to 
the subtracting circuit 5. In the subtracting circuit 5, a 
difference between the pixels of the actual video signal 
and the prediction signal is formed. 

The motion vector detecting circuit 6 detects a mo- 
tion vector for motion compensation from the output vid- 
eo signal of the A/D converter 3, namely, the digital video 
signal before the camera shake correction. In the em- 
bodiment of the invention, the motion vector Vhv and an 
evaluation value Pij are supplied to the camera shake 
detecting circuit 12 and, as will be described below, a 
camera shake motion vector is obtained and is supplied 
to the correction signal generating circuit 13, thereby 
forming a correction signal. The correction signal is sup- 
plied to the camera shake correcting circuit 4, so that 
the camera shake can be corrected. 

In order to record and reproduce the image signal, 
a construction shown in Fig. 4 is used. In Fig. 4, an en- 
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coded output is supplied from the encoder (in the exam- 
ple of Fig. 3, ADRC encoder 8) for motion compensation 
predictive encoding to an input terminal designated by 
reference numeral 21. The encoded output is supplied 
to an encoder 22 for variable length encoding and is var- 
iable length encoded. The variable length encoded out- 
put is supplied to a recording buffer 23. The recording 
buffer 23 feeds back a signal indicative of a buffer 
amount to the front stage. Thus, an amount of recording 
data for a predetermined period of time is controlled to 
a predetermined value. For example, an amount of 
transmission data can be controlled by changing the 
number of quantization bits of the ADRC encoding. 

Although a buffering circuit for feed-back control is 
used in the example, it is also possible to use a con- 
struction in which a plurality of bits (for example, zero 
bit, one bit, two bits, three bits, four bits) is prepared as 
the number of quantization bits in the ADRC encoding, 
and a buffering for feed-forward control is used such that 
an amount of generation data for a predetermined peri- 
od of time is estimated, and the number of quantization 
bits is selected every block so as not to exceed a target 
data amount is executed. An output of the recording 
buffer 23 is supplied to a multiplexer 24 and audio data, 
data for control, and the like are multiplexed with that 
output. 

An output of the multiplexer 24 is supplied to a re- 
cording processing circuit 25. Processes such as error 
correction encoding, channel modulation, and the like 
for the recording data are performed in the recording 
processing circuit 25. The recording data from the re- 
cording processing circuit 25 is recorded to a recording 
medium through a recording amplifier 26 and a record- 
ing side terminal R of a recording/reproduction change- 
over switch 27. As a recording medium, a magnetic 
tape, a recordable optical disc, or the like can be used. 

Reproduction data reproduced from the recording 
medium is supplied to a reproduction processing circuit 
29 through a reproducing side terminal P of the record- 
ing/reproduction change-over switch 27 and a reproduc- 
ing amplifier 28. The reproduction processing circuit 29 
performs processes such as demodulation of the chan- 
nel modulation, error correction, and the like. An output 
of the reproduction processing circuit 29 is supplied to 
a demultiplexer 30 and an audio signal, data for control, 
and the like are separated. Reproduction data corre- 
sponding to a video signal from the demultiplexer 30 is 
supplied to a decoder 31 of the variable length encoding, 
so that the variable length encoded data is decoded. 
.The reproduction data extracted to an output terminal 
32 is supplied to a decoder of a high efficient encoding 
such as an ADRC decoder or the like. 

The ADRC is one of the high efficiency encoding 
methods proposed by the inventors of the present in- 
vention. An example of a construction of the ADRC en- 
coder 8 is shown in Fig. 5. A digital video signal is sup- 
plied from an input terminal 34 to a block segmenting 
circuit 35. Data having the order of every block obtained 



by segmenting one picture plane is formed by the block 
segmenting circuit 35. Output data of the block seg- 
menting circuit 35 is supplied to a detecting circuit 36 
and a delay circuit 37. 

$ The detecting circuit 36 detects a maximum value 

MAX and a minimum value MIN of a plurality of pixels 
of each block. The delay circuit 37 delays the digital vid- 
eo signal by a time which is required for such detection. 
The detected maximum value MAX and minimum value 

10 MIN are supplied to a subtracting circuit 38 and a dy- 
namic range DR shown by (MAX - MIN = DR) is obtained 
from the subtracting circuit 38. The digital video signal 
from the delay circuit 37 and the minimum value MIN 
are supplied to a subtracting circuit 39, so that corrected 

'5 pixel data in which the minimum value has been elimi- 
nated is obtained as an output of the subtracting circuit 
39. A normalization is performed by eliminating the min- 
imum value MIN which is shared by the pixels in the 
block. 

20 An output of the subtracting circuit 39 and the de- 
tected dynamic range DR are supplied to a quantizing 
circuit 40. In the quantizing circuit 40, as shown in Fig. 
7, the output of the subtracting circuit 39 is again quan- 
tized by a quantizing step of dividing the dynamic range 

25 DR into 2n equal parts (n: the number of quantization 
bits and n = 2 in Fig. 7). Since the pixels which are spa- 
tially neighboring have a strong correlation, even if the 
number of quantization bits n is set to be smaller than 
the original number of quantization bits (for example, 8 

^o bits), a deterioration in decoded image is small, so that 
the transmission data amount can be compressed. An 
encoded output DT from the quantizing circuit 40 is 
transmitted and the dynamic range DR of every block 
and the minimum value MIN are transmitted as addition- 
's al information. 

Fig. 6 shows a construction of an example of the 
ADRC decoder 9. The dynamic range DR and encoded 
output DT from the ADRC encoder 8 are supplied to an 
inverse quantizing circuit 42. The inverse quantizing cir- 

40 cuit 42 multiplies a quantizing step determined by the 
dynamic range DR and the number (n) of quantization 
bits to the encoded output and rounds a multiplexed out- 
put to an integer, thereby generating reconstructing lev- 
els. As shown in Fig. 7, medians within the ranges of the 

45 levels upon quantization are set to reconstructing levels 
(representative levels) L0, L1, L2, and L3. 

The reconstructing level from the inverse quantizing 
circuit 42 and the minimum value MIN are added by an 
adding circuit 43, thereby obtaining a decoded value. 

50 The decoded value is supplied to a block desegmenting 
circuit 44 and the order of pixel data is returned to a 
raster order. A decoded video signal is extracted to an 
output terminal 45 of the block desegmenting circuit 45. 
The motion vector is detected by a block matching 

55 method in the motion vector detecting circuit 6. Such a 
detecting process is as schematically shown in Fig. 8. 
A check block By of a reference frame, for example a 
previous frame Fn-1, is moved within a predetermined 
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search area and a check block which most coincides 
with a reference block Bx of a present frame Fn is de- 
tected, thereby obtaining the motion vector. Therefore, 
the motion vector is obtained every block. As an evalu- 
ation value indicative of a degree of coincidence, be- 
tween a plurality of pixels in the reference block Bx and 
a plurality of pixels in the check block By, a subtraction 
is performed between the values of the pixels at the 
same spatial position, thereby obtaining frame differ- 
ences. A sum of the absolute values of the frame differ- 
ences can be used. A sum of the squares of the frame 
differences can also be used in place of the sum of the 
absolute values of the frame differences. 

Fig. 9 shows a construction of an example of the 
motion vector detecting circuit 6. In Fig. 9 : reference nu- 
meral 51 denotes an input terminal of the image data of 
the present frame. The image data is stored in a present 
frame memory 53. Reference numeral 52 denotes an 
input terminal of the image data of the previous frame 
and the image data is stored in a previous frame mem- 
ory 54. 

The reading/writing operations of the present frame 
memory 53 and previous frame memory 54 are control- 
led by a controller 55 Pixel data of the reference block 
of the present frame is read out from the present frame 
memory 53. Pixel data of the check block of the previous 
frame is read out from the previous frame memory 54. 
An address moving circuit 56 is provided in association 
with the previous frame memory 54. The controller 55 
controls the address moving circuit 56, so that the posi- 
tion of the check block is changed within a search area 
by one pixel step at a time. 

An output of the present frame memory 53 and an 
output of the previous frame memory 54 are supplied to 
a difference detecting circuit 57, thereby detecting a dif- 
ference (frame difference) of every pixel. An output of 
the difference detecting circuit 57 is converted to an ab- 
solute value by an absolute value forming circuit 58. The 
absolute value is supplied to an accumulating circuit 59. 
The accumulating circuit 59 accumulates the absolute 
value differences which occurred in one block. An output 
(frame difference absolute value sum) of the circuit 59 
is supplied to a judging circuit 60 as an evaluation value. 
The judging circuit 60 detects the motion vector from the 
absolute value sum of the differences which occur when 
the check block is moved within the search area. That 
is, the position of the check block which causes the sum 
of the absolute values to have the minimum differences 
is detected as a motion vector. The detected motion vec- 
. tor is extracted to an output terminal 61 . 

According to the embodiment of the invention, the 
motion vector for motion compensation detected as 
mentioned above is also used to correct the camera 
shake. The motion vector for camera shake correction 
shows a motion of the whole picture plane or a relatively 
large block. As an existing method, as shown in Fig. 10, 
there has been known a method of detecting the motion 
vector for camera shake correction. 



That is, the present frame Fn is divided into search 
areas Sx, thereby obtaining the absolute values of the 
frame differences between a plurality of pixels in each 
search area Sx and a representative point pixel Ry (pixel 
s at the center position of the search area) of the previous 
frame Fn-1. Further, the frame difference absolute val- 
ues obtained in each search area Sx are integrated at 
the same spatial position, thereby forming an evaluation 
value table having the same size as that of the search 
10 area Sx. The minimum value in the evaluation value ta- 
ble is detected, thereby obtaining the motion vector of 
the whole picture plane. 

As mentioned above, the motion vector to be ob- 
tained by the motion vector detecting circuit 6 is ob- 
15 tained on a block unit basis. On the other hand, the mo- 
tion vector which is necessary to correct the camera 
shake is obtained by the whole picture plane or on a unit 
basis of a relatively large block. Since there is such a 
difference, the camera shake detecting circuit 12 re- 
20 ceives the frame difference absolute value sum Pij and 
the motion vector Vhv from the motion vector detecting 
circuit 6 and forms the motion vector for camera shake 
correction. 

Fig. 11 shows a construction of an example of the 

25 camera shake detecting circuit 12 and camera shake 
correcting circuit 4. The frame difference absolute value 
sum Pij from the motion vector detecting circuit 6 is sup- 
plied to an integrating circuit 71. The frame difference 
absolute value sum Pij will now be described with refer- 

30 ence to Fig. 12. As described with reference to Fig. 8, 
the absolute value sum of the frame differences be- 
tween the reference block Bx and check block By is cal- 
culated at each position of the check block By in the 
search area. As an example, now assuming that the 

35 search area is set to ±4 in the horizontal direction and 
±3 in the vertical direction, the frame difference absolute 
value sum between the reference block and (9x7 = 63) 
check blocks is calculated. 

Fig. 1 2 shows a state in which the frame difference 

^0 absolute value sum with respect to one certain refer- 
ence block Bx of the present frame is arranged in cor- 
respondence to the positions of the check blocks. A 
frame difference absolute value sum P43 at the center 
position (i = 4, j = 3) is the sum which is obtained when 

45 the reference block Bx and check block By exist at the 
same spatial position. The frame difference absolute 
value sum Pij is integrated by the integrating circuit 71 
every reference block, thereby forming an evaluation 
value IPhv. As an example, it is now assumed that a 

50 block size is set to (16 x 16) and an effective picture 
plane size of one frame is set to (704 pixels x 480 lines). 
As shown in Fig. 13, one frame is divided into (44 x 30) 
blocks, so that evaluation values IPhv of a number that 
is equal to the number of blocks are obtained. 

55 The evaluation value IPhv reflects a distribution of 
the levels of the images in the block. When the image 
in the search area has a flat level distribution, the level 
of the frame difference absolute value sum decreases 
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and the level of the evaluation value IPhv also decreas- 
es. On the other hand, when the level distribution is not 
flat but includes an edge or the like, the level of the eval- 
uation value IPhv also increases. In the embodiment, 
the level distribution of images is used as an activity. 
The flat level distribution is set to a low activity. The ac- 
tivity is not limited to a spatial inclination but also in- 
cludes a frequency component or the like in the frequen- 
cy conversion. 

The evaluation value IPhv and the motion vector 
Vhv are supplied to a camera shake judging circuit 72. 
As shown in a flowchart of Fig. 14, the camera shake 
judging circuit 72 performs a camera shake judgment. 
As initial values, (h = 0, v = 0) are set (step 81 ) and Vhv 
and IPhv are respectively compared with threshold val- 
ues TH 1 and TH2 in step 82. When V00 < TH 1 and IP00 
> TH2, it is judged that the camera stands still (namely, 
there is no camera shake) (step 83). 

When the conditions in step 82 are satisfied, it 
means that the block 00 corresponds to an image at a 
position in which the level change of the edge or the like 
is discontinuous (IP00 > TH2) and it is detected that 
there is no motion with respect to the block 00 (V00 < 
TH 1 ). When the camera shake occurs, the whole picture 
plane moves. Therefore, when at least one still block 
exists, it can be judged that there is no camera shake. 

When the conditions in step 82 are not satisfied, the 
processing routine advances to step 84 and a check is 
made to see if a value of h is the maximum value (43) 
in the horizontal direction. If NO, in step 85 the value of 
h is increased by +1 and the processing routine is re- 
turned to step 82. As mentioned above, a processing 
routine is shifted from the process with respect to the 
block 00 to an adjacent block 10 and whether the block 
10 is a still block or not is judged. When the processes 
with respect to all of the blocks in which v = 0 are fin- 
ished, the processing routine advances from step 84 to 
step 86. 

In step 86, whether a value of v is equal to 29 or not 
is determined. If NO, the processing routine advances 
to step 87. In step 87, the value of v is increased by +1 
and the processing routine is returned to step 82. When 
the value of v reaches 29, the processes of all of the (44 
x 30) blocks are completed. When no still block is de- 
tected among any of the blocks, the motion of the whole 
picture plane, namely camera shake, is discriminated. 
In step 88, the processing routine advances to a process 
for the motion vector detection every macroblock. In the 
construction of Fig. 11 , a control signal of the judgment 
result is generated from the camera shake judging cir- 
cuit 72 to a macroblock segmenting circuit 73. 

The macroblock segmenting circuit 73 separates 
the motion vector Vhv from the motion vector detecting 
circuit 6 every macroblock. In this instance, as shown in 
Fig. 15, the picture plane of one frame is divided into 
four portions in the horizontal direction and is further di- 
vided into three portions in the vertical direction, thereby 
forming 12 macroblocks each of a size of (11 x 10) 



blocks. In order to obtain a plurality of macroblocks, the 
method of dividing one picture plane is not limited to 
such an example of the macroblocks but various meth- 
ods can be used. It is, however, necessary to generate 
s the macroblocks such that at least some are not neigh- 
boring each other. Therefore, a method of dividing the 
picture plane into four equivalent portions must not be 
used. The motion vector is supplied to motion vector de- 
tecting circuits 74 0 , 74j, and 74-, 2 every macroblock. 

10 The motion vector detecting circuit 740 detects a 
synthesis motion vector MV00 by adding the motion 
vectors of the blocks included in a macroblock MB00. 
Similarly, the motion vector detecting circuits 74-,, 
74 2 , .... and 74 12 detect the synthesis motion vectors 

is MV10, MV20, MV32 of the macroblocks, respective- 
ly. The detected synthesis motion vector MV00 to MV32 
are supplied to an integrated vector forming circuit 75. 

The integrated vector forming circuit 75 generates 
the motion vector (integrated vector) of the whole picture 

20 plane by integrating the 12 synthesis motion vectors. 
The integrated vector is supplied to the correction signal 
generating circuit 13. The integrated vector is detected 
from the motion between the frames and is not the same 
as a correction amount of the camera shake. For exam- 

25 pie, for a continuous period of three frames, when cam- 
era shake in the same direction occurs for the period of 
the second frame and the period of the third frame, an 
integrated vector V1 between the first frame and the 
next frame is obtained and an integrated vector V2 be- 

30 tween the second and third frames is obtained. Although 
it is sufficient that a correction amount for the second 
frame is equal to V1, a correction amount for the third 
frame needs to be equal to (V1 + V2). The correction 
signal generating circuit 13 generates a correction 

35 amount obtained by integrating the integrated vector as 
an example. A camera shake correction signal from the 
correction signal generating circuit 1 3 is supplied to the 
correcting circuit 4, thereby correcting the camera shake 
of the input image data. 

40 The integrated vector forming circuit 75 forms the 
integrated vector in accordance with a flow shown in Fig. 
16. The motion vector of each macroblock is inputted 
(step 91 ) and a check is made to see if the motion vec- 
tors of all of the macroblocks are the same (step 92). If 

->5 YES, such a motion vector is outputted as an integrated 
vector (step 93). In a manner similar to the above, the 
term "same" includes a certain degree of tolerance. 

When the condition of step 92 is not satisfied, the 
processing routine advances to judging step 94 and a 

50 check is made to see whether or not the motion vectors 
are the same with respect to a plurality of macroblocks 
which are spatially separated. In other words, a check 
is made to see whether the macroblocks having the 
same motion vector have spatially been separated in the 

55 whole one frame or not. A motion vector which satisfies 
step 94 is outputted as an integrated vector (step 95). 
The words "is spatially separated" means that the mac- 
roblocks [for example, (MB00, MB10, MB01, MB11)] 
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which are neighboring in the vertical, lateral, and oblique 
directions are eliminated. Therefore, the number of 
macroblocks having the spatial separation and the 
same motion vector and the like are properly set in con- 
sideration of a necessary detecting precision or the like. 
An erroneous judgment such that a motion of an object 
of a relatively large area is regarded as camera shake 
can be eliminated by the discriminating step 94. 

If NO in step 94, step 96 follows and the numbers 
of macroblocks having the same vector are totaled. A 
weight is added to each number of the macroblocks in 
step 97. Such a weighting process is executed to give 
a priority to the number of blocks concerning macrob- 
locks in the peripheral portions of the picture plane over 
the number of blocks concerning macroblocks near the 
non-peripheral portions of the picture plane. For exam- 
ple, a weight coefficient of 1 .5 is multiplied to the number 
of blocks relating to the peripheral portions and a weight 
coefficient of 1 is multiplied to the number of blocks re- 
lating to the portions near the non-peripheral portions. 
The weighting process in step 97 is performed to the 
motion of the object of a relatively large area in consid- 
eration of a fact that a number of macroblocks exist near 
the center. 

The processing routine advances to step 98 and a 
check is made to see if there is a maximum value among 
the weighted numbers of blocks. If YES, the motion vec- 
tor of the macroblock of the maximum value is outputted 
as an integrated vector (step 99). If the numbers of 
blocks are equal or are almost equal step 100 follows. 
With regard to the number of macroblocks, a majority 
decision is performed in step 100. The motion vector 
which has been judged as a majority is outputted as an 
integrated vector. The majority judgment has a problem 
in terms of a precision when the integrated vector is gen- 
erated. In case of attaching importance to the precision, 
such a judgment is omitted and it is also possible to proc- 
ess by regarding such that the integrated vector cannot 
be detected. 

By the above process, since the motion vector is 
integrated to the motion vector of the whole picture 
plane by using a difference between the characteristics 
of the motion object and the camera shake, the detected 
camera shake motion vector is not influenced by the mo- 
tion of the target object in the picture plane, so that the 
motion vector has a high precision. 

Explanation will now be continued by returning to 
Fig. 11 . The integrated vector indicative of the motion of 
the whole picture plane is supplied to the correction sig- 
nal generating circuit 13 and a correction signal is 
formed by integration. The correction signal is supplied 
to the camera shake correcting circuit 4 and the camera 
shake is corrected. The camera shake correcting circuit 
4 comprises: a memory 101 in which image data from 
the delay circuit 14 is written; a peripheral memory 102; 
a selector 1 03 to select read outputs of the memory 101 
and peripheral memory 102; an address control circuit 
105 to control addresses in the memory 101 and periph- 



eral memory 102; and a selection signal generating cir- 
cuit 106 for generating a selection signal to control the 
selector 103. A video signal in which the camera shake 
has been corrected is taken out to an output terminal 
5 104. 

The memory 101 is, for example, a frame memory 
and its read address is controlled by the correction sig- 
nal. Therefore, image data moved in accordance with 
the correction signal is read out from the memory 101. 

10 The data selected by the selector 103 is written into the 
peripheral memory 102. In response to a selection sig- 
nal from the selection signal generating circuit 106, the 
selector 103 selects the camera shake corrected image 
data from the memory 101 and the peripheral data 

is stored in the peripheral memory 102. 

In Fig. 17A, a peripheral portion (region on the out- 
side of an alternate long and short dash line) 108 of an 
image (shown by its picture frame 107) of one frame 
stored in the memory 101 is stored into the peripheral 

20 memory 1 02. A width of the peripheral portion 1 08 is set 
in consideration of a range of the camera shake correc- 
tion. For example, it is set to a width of about 10 to 20 
% of the image of one frame in each of the horizontal 
and vertical directions. In Fig. 1 73, as shown in a picture 

25 frame 107a, when the image which ought to be located 
at a position in Fig. 17A is moved, for example to the 
right of the drawing, by camera shake, the whole image 
is corrected to a position shown by a broken line on the 
basis of a camera shake correction amount V. In the 

30 case of the camera shake correction, an image of a por- 
tion 109 which does not inherently exist and is shown 
by a hatched region on the left side of the image after 
the movement drops out in the photographed image. 
The image at the corresponding position stored in the 

35 peripheral memory 102 is substituted for the dropped 
portion 109. Such a substitution is executed by an ad- 
dress control by the address control circuit 105 and the 
switching operation of the selector 103. The peripheral 
image data other than the dropped portion 109 in the 

40 photographed image data is written into the peripheral 
memory 102. The contents in the peripheral memory 
102 are updated. 

As mentioned above, since the drop-out image 
which occurs due to the camera shake correction is re- 

^5 placed by the peripheral image stored in the peripheral 
memory, different from a process for enlarging the im- 
age, a deterioration of a resolution of the image can be 
prevented. 

According to the embodiment of the invention men- 
50 tioned above, the motion vector to detect the camera 
shake motion vector is obtained by processing the video 
signal before the camera shake correction. A process 
such as a motion compensation or the like is executed 
to the video signal which was camera shake corrected 
55 by the camera shake correcting circuit 4. Therefore, the 
motion compensation cannot be performed by the mo- 
tion vector detected by the motion vector detecting cir- 
cuit 6. To solve such a problem, the motion vector cor- 
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recting circuit 15 is provided. Camera shake compo- 
nents included in the motion vector are eliminated by 
the motion vector correcting circuit 1 5. 

The process in the motion vector correcting circuit 
1 5 will now be described with reference to Figs. 1 8A and 
18B. Fig. 18A shows that the motion vectors V00, V01, 
V10, V11, etc. obtained in each block include the 
camera shake vector components. When there is no 
camera shake, the positions of those blocks exist at the 
positions shown by broken lines. 

Considering the block of the motion vector V00, as 
shown in Fig. 1 8B, when the camera shake components 
are eliminated the position of this block exists at the po- 
sition shown by the broken line. With respect to the block 
existing at the position of the broken line, the number of 
pixels included in the block at the position where the mo- 
tion vector was detected and the number of pixels in- 
cluded in the other peripheral blocks are obtained. In 
Figs. 18A and 18B, the whole number of pixels of the 
block is set to n and at the positions shown by broken 
lines, n1 , n2 : n3, and n4 indicate the numbers of pixels 
included in the blocks. A motion vector V00' after the 
correction is calculated by a weighted mean method 
shown by the following equation. The motion vectors of 
the other blocks are also corrected in a manner similar 
to that mentioned above. 

V00' = (n1/n)*V00 + (n2/n)-V10 + (n3/n)-V01 + 
(n4/n).V1 1 

To obtain the numbers (n1 to n4) of pixels included 
in those plurality of blocks, a projection component to 
the x axis of the camera shake correction and a projec- 
tion component to the y axis are used. The motion vector 
can be corrected as mentioned above. The weighted 
mean method is one of the possible correcting methods. 
Other correcting methods can also be used. 

Fig, 1 9 shows another embodiment of the invention. 
In the embodiment shown in Fig. 3, an interframe differ- 
ence from the subtracting circuit 5 is compressed by the 
ADRC. However, an encoding of the MPEG (Moving 
Picture Experts Group) standard is used in the Fig. 19 
embodiment. According to the MPEG standard, a com- 
pression using spatial correlation is executed by a DCT 
(Discrete Cosine Transform) circuit 1 6a and a quantizer 
1 7a and a bidirectional motion compensation interframe 
prediction is executed. 

The bidirectional predictive encoding comprises an 
intraframe predictive encoding, a forward direction pre- 
dictive encoding, and a reverse direction predictive en- 
coding. The forward direction predictive encoding is an 
interframe predictive encoding for predicting the present 
image from a past image. The reverse direction predic- 
tive encoding is an interframe predictive encoding for 
predicting the present image from a future image. Fur- 
ther, an interpolative interframe predictive encoding by 
the prediction in both of the front and back directions is 



also performed. In correspondence to those predictive 
encoding methods, picture types (I picture, P picture, B 
picture) are specified. 

For the bidirectional motion compensation, a frame 
5 memory 18 is added to the motion vector detecting cir- 
cuit 6. A motion compensating circuit 7r in the reverse 
direction, an interpolation motion compensating circuit 
7i, and a forward direction compensating circuit 7f exe- 
cute respective motion compensating processes by us- 

10 jng the past, present, and future predictive images from 
frame memories 11a and 11b, respectively. An inverse 
quantizer 16b, an inverse DCT circuit 17b, and an add- 
ing circuit 10 for adding outputs of the inverse DCT cir- 
cuit 17b and a signal transmitted through a switching 
circuit are provided for a local decoding. A switching cir- 
cuit for selecting either one of a difference signal from 
the subtracting circuit 5 and a signal which is transmitted 
without passing through the subtracting circuit 5 is pro- 
vided. Those switching circuits are controlled by control 

20 signals S1, S2, and S3 in accordance with the picture 
types mentioned above. The control signals S1 , S2 ; and 
S3 are generated from an encoding control circuit 19. 

According to the Fig. 19 embodiment of the inven- 
tion, in a manner similar to the foregoing embodiment, 

25 the motion vector Vhv from the motion vector detecting 
circuit 6 and the evaluation value Ptj are supplied to the 
camera shake detecting circuit 1 2, a camera shake mo- 
tion vector is obtained and supplied to the correction sig- 
nal generating circuit 13, and a correction signal is 

30 formed and supplied to the camera shake correcting cir- 
cuit 4, thereby correcting the camera shake. To correct 
the camera shake motion vector, the motion vector cor- 
recting circuit 15 is provided. 

Fig. 20 shows a construction on the recording side 

35 of still another embodiment of the invention. In this em- 
bodiment, the correction signal for camera shake cor- 
rection is generated on the recording side and the cam- 
era shake correction is performed on the reproducing 
side illustrated in Fig. 21. Therefore, as shown in Fig. 

40 20, although the camera shake detecting circuit 1 2 and 
correction signal generating circuit 13 are provided on 
the recording side, the camera shake correcting circuit 
4 is not provided. An encoded output from the ADRC 
encoder 8, the motion vector, and the camera shake cor- 

45 rection signal are supplied to a frame segmenting circuit 
20, so that recording data is formed. The recording data 
is recorded onto a recording medium through the re- 
cording system. 

As shown in Fig. 21, on the reproducing side, the 

so reproduction data formed by the reproducing system is 
supplied from an input terminal 62 to a frame deseg- 
menting circuit 63, by which an ADRC encoded output, 
the camera shake vector Vhv, and the correction signal 
are separated. The difference signal is decoded by an 

55 ADRC decoder 64 and a decoded difference signal is 
supplied to an adding circuit 65. An output of the adding 
circuit 65 is supplied to a prediction memory 66 and a 
motion compensation similar to that on the recording 
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side is performed by the motion vector Vhv. 

The reproduction image signal from the adding cir- 
cuit 65 is supplied to the camera shake correcting circuit 
4. The camera shake correcting circuit 4 executes the 
camera shake correction by the correction signal sepa- 5 
rated from the reproduction signal. The digital image sig- 
nal from the camera shake correcting circuit 4 is sup- 
plied to a D/A converter 67 and the reproduction image 
signal is taken out to an output terminal 68. It is also 
possible to provide selecting means for selecting either io 
one of a mode to make the camera shake correction op- 
erative and a mode to make it inoperative. Although the 
camera shake correction signal has been recorded in 
the embodiment, in place of the camera shake correc- 
tion signal, it is also possible to record an evaluation val- 15 
ue Pij and to form the camera shake correction signal 
by using the motion vector and the evaluation value on 
the reproducing side. 

The Fig. 20 embodiment of the invention in which 
the camera shake correction signal has been recorded 20 
can be also applied to the case of using the MPEG 
standard for encoding the data. 

Although the peripheral memory has been provided 
to compensate the drop-out of images upon camera 
shake correction, the drop-out of the images can also 25 
be compensated by a background memory. The back- 
ground memory is a memory in which images which 
hardly change in a change in image between the frames 
are selectively written. By using the background mem- 
ory, an uncovered background when executing the mo- 30 
tion compensation can be prevented or a transmission 
data amount can be compressed by thinning out the 
background data to be transmitted. The compression 
encoding method according to such a method has been 
proposed by the inventors of the present invention and 35 
disclosed in, for example, JP-B-7-97754. By using a 
memory having a capacity larger than that of the frame 
memory as a background memory, it can be used to ac- 
cumulate images such as peripheral image due to cam- 
era shake : image by a pan or a tilt, and the like which 40 
cannot be enclosed in one frame. 

Fig. 22 is a diagram for schematically explaining the 
image data to be accumulated into the background 
memory and shows that, for example, when a tennis 
player who plays on a court is photographed, an image 45 
which sequentially changes and a background image 
(specifically speaking, tennis court) in the photographed 
image are accumulated into the background memory. 
That is, the image excluding the moving player is accu- 
mulated in a wide range in the background memory. so 

Fig. 23 is a diagram showing an embodiment of the 
invention using the background memory. In Fig. 23, the 
component elements corresponding to those in the em- 
bodiment shown in Fig. 3 are designated by the same 
reference numerals and their descriptions are omitted. 55 
The image signal which was local decoded and motion 
compensated is inputted from the adding circuit 10 to a 
camera shake correcting circuit 111. The camera shake 



correcting circuit 111 executes the camera shake cor- 
rection, for example, on a block unit basis. Namely, for 
example, the block position is corrected by the camera 
shake correction signal from the correction signal gen- 
erating circuit 13. The image signal which is generated 
from the camera shake correcting circuit 11 1 is inputted 
to a background memory 11 2 through a comparator 11 3. 

On the other hand, the image data at the position 
corresponding to the corrected block position is read out 
from the background memory 1 1 2. The comparator 1 1 3 
calculates a difference value between the block includ- 
ing the local decoded image data and the image data at 
the corresponding position from the background mem- 
ory 1 1 2 every pixel. The absolute value of the difference 
value is accumulated in one block. The accumulated 
value of the differences of the blocks is compared with 
a predetermined threshold value by the comparator 1 1 3. 
A comparison output of the comparator 113 is inputted 
to an updating circuit 114. 

When the accumulated value of the differences is 
smaller than the threshold value, the updating control 
circuit 114 judges that the block is a block of a back- 
ground image. When the block is decided to be the block 
of the background image, the data in the background 
memory 112 at the position corresponding to such a 
block is updated by the image block data. Upon updat- 
ing, it is desirable to gradually update the contents in 
the background memory 112 rather than the case of up- 
dating the data in a lump. When the accumulated value 
of the differences is equal to or larger than the threshold 
value, it is determined that the data of the block includes 
the image information other than the background image 
and is not used for updating of the contents in the back- 
ground memory 112. Further, when a memory of a ca- 
pacity larger than that of the frame memory is used as 
a background memory 112 as shown in the example of 
Fig. 22, new background data which is not filled with the 
data of the frames of up to the previous frame is sequen- 
tially accumulated. 

By performing the motion compensation by selec- 
tively using the data accumulated in the background 
memory 112 as mentioned above and the data in the 
ordinary frame memory, the dropped images can be 
compensated. The image data in the background mem- 
ory 112 is supplied to a motion vector detecting circuit 
6' with a construction shown in Fig. 24. 

To the construction having the motion vector detect- 
ing circuit 6 shown in Fig. 9 mentioned above, a differ- 
ence detecting circuit 57', an absolute value forming cir- 
cuit 58\ and an accumulating circuit 59' are added. An 
output of the accumulating circuit 59' is supplied to a 
judging circuit 60' together with an output of the accu- 
mulating circuit 59. The judging circuit 60' outputs a mo- 
tion vector and a memory flag to output terminals 61 and 
6V, respectively. 

The difference detecting circuit 57" detects a differ- 
ence of the pixel data at the same position between the 
output data of the background memory 1 1 2 and the out- 
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put data of the present frame memory 53. The difference 
value is converted intoan absolute value by the absolute 
value forming circuit 58' and is accumulated by the ac- 
cumulating circuit 59'. The judging circuit 60' receives 
an accumulation output (accumulated value of the dif- 
ferences between the present frame and the previous 
frame) from the accumulating circuit 59 and the forego- 
ing accumulation output from the accumulating circuit 
59'. The judging circuit 60' outputs a more proper motion 
vector to the output terminal 61 on the basis of one ac- 
cumulated output. The judging circuit 60' outputs the 
memory flag indicative of the selected one of the two 
accumulated outputs to the output terminal 61 '. 

As shown in Fig. 23, the motion vector is supplied 
to the motion compensating circuit 7 through the motion 
vector correcting circuit 1 5. The memory flag is supplied 
to the motion compensating circuit 7. The motion vector 
and the memory flag are transmitted or recorded togeth- 
er with the encoded data. One of the image data accu- 
mulated in the frame memory 11 and the background 
image data accumulated in the background memory 1 1 2 
is selected by the memory flag. A motion compensating 
process is executed to the selected image data. 

Although the encoder side in the Fig. 23 embodi- 
ment has been described above, the background mem- 
ory is also provided on the decoder side and either one 
of the image in the frame memory and the image in the 
background memory is selected in accordance with the 
memory flag in a manner similar to the decoder side. 

According to the invention, since the camera shake 
motion vector is detected from the motion vector formed 
for motion compensation, the scale of the hardware can 
be reduced. 

The present invention is not limited to the foregoing 
embodiments but many modifications and variations are 
possible within the scope of the appended claims of the 
invention. 



Claims 

1 . An image signal processing apparatus having func- 
tions for 

dividing image data of one picture plane into a 
plurality of blocks, 

detecting a motion vector corresponding to a 
position of the block which most coincides from 
the image data of one or a few frames before 
with respect to each block, and 
compressing the image data by using said mo- 
tion vector, comprising: 

means (6) for detecting the motion vector of 
each block; 

means (12) for detecting a camera shake mo- 
tion vector from the motion vector of each of 
said blocks; 

means (4) for correcting an input image signal 



on the basis of said camera shake motion vec- 
tor; 

means (15) for obtaining a motion vector in 
which the motion vector of each of said blocks 
5 has been corrected by said camera shake mo- 

tion vector; 

means (7) for performing a motion compensa- 
tion by said corrected motion vector; and 
means (5, 8) for executing a compressing proc- 
to ess on said image data which was camera 

shake corrected and said image data which 
was motion compensated. 

2. An apparatus according to claim 1, wherein said 
*5 camera shake motion vector detecting means (1 2) 

has: 

means for evaluating an activity of the image 
data of each block; and 
20 means for detecting the same motion vector at 

positions which are spatially discontinuous. 

3. An apparatus according to claim 1 , wherein said 
camera shake motion vector detecting means (12) 

2S has still block judging means (82) comprising 

means for evaluating an activity of the image 
data of each block and 

means for detecting the block of a motion vector 
30 o, 

and wherein when a still vector is detected, the 
camera shake correction is not performed. 

4. An apparatus according to claim 1 , wherein said 
35 camera shake correcting means (4) has: 

first image information memory means (1 01 ) to 
which image data of a target to be subjected to 
the camera shake correction is inputted; 

-to second image information memory means 

(102) in which image data around frame image 
data has been accumulated; and 
selecting means (103) for selecting one of the 
image data outputted from said first and second 

4 $ image information memory means in accord- 

ance with said camera shake motion vector. 

5. An apparatus according to claim 1 , wherein said 
motion vector detecting means comprises: 

so 

input image information memory means (53) to 
which present image data is inputted; 
first comparison image information memory 
(54) means in which the image data of one or 
55 a few frames before has been stored; 

second comparison image information memory 
(112) means in which background image data 
existing in a plurality of frames has been stored; 
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first means (57-59) for detecting the motion 
vector by said present image data and said pre- 
vious image data; 

second means (57'-59') for detecting the mo- 
tion vector by said present image data and said 5 
background image data: and 
means (60') for selecting the motion vector hav- 
ing smaller error data in said two detected mo- 
tion vectors and for outputting said motion vec- 
tor (61 ) and a memory instruction flag (61 ') in- io 
dicative of either one of the first and second 
comparison image information memory means 
corresponding to said selected motion vector. 

6. An apparatus according to claim 5, wherein said 7 $ 
camera shake correcting means selects and out- 
puts either one of 

the input image data to be subjected to the cam- 
era shake correction and 20 
said background image data which is obtained 
from said second comparison image informa- 
tion memory means 

in accordance with said camera shake motion 

vector. - -~ - - 25 9. 

7. An image signal processing apparatus having func- 
tions for 

dividing image data of one picture plane into a 30 
plurality of blocks, 

forming an evaluation value table based on a 
difference between the present image data and 
the image data of the block which is one or a 
few frames before every pixel with respect to 35 
each block, 

detecting coordinates of a minimum value in 
said evaluation value table, 
detecting a motion vector on the basis of said 
coordinates, and 40 
compressing the image data by using said mo- 
tion vector, comprising: 

means (12) for detecting a camera shake mo- 
tion vector on the basis of said evaluation value 
table and said motion vector: 
means (4) for correcting an input image signal 
on the basis of said camera shake motion vec- 
tor; 

means (15) for correcting said motion vector by 
using said camera shake motion vector; so 
means (7) for performing a motion compensa- 10. 
tion by said corrected motion vector; and 
means (5, 8) for executing a compressing proc- 
ess by said image data which was camera 
shake corrected and said image data which 55 
was motion compensated. 

8. An image signal recording apparatus having func- 



tions for 

dividing image data of one picture plane into a 
plurality of blocks, 

detecting a motion vector corresponding to a 
position of the block which most coincides from 
the image data which is one or a few frames 
before with respect to each block, and 
compressing the image data by using said mo- 
tion vector, comprising: 

means (6) for detecting the motion vector of 
each block: 

means (12) for detecting a camera shake mo- 
tion vector from the motion vector of each of 
said blocks; 

means (7) for performing a motion compensa- 
tion by said motion vector; and 
means (5, 8) for executing a compressing proc- 
ess by input image data and the image data 
which was motion compensated, 
wherein said camera shake motion vector is re- 
corded together with said compressed image 
signal. 

An apparatus according to claim 8, wherein said 
motion vector detecting means comprises: 

input image information memory means (53) to 
which present image data is inputted: 
first comparison image information memory 
(54) means in which the image data of one or 
a few frames before has been accumulated; 
second comparison image information memory 
(112) means in which background image data 
existing in a plurality of frames has been stored; 
first means (57-59) for detecting the motion 
vector by said present image data and said pre- 
vious image data; 

second means (57'-59') for detecting the mo- 
tion vector by said present image data and said 
background image data; and 
means (60*) for selecting the motion vector hav- 
ing smaller error data in said two detected mo- 
tion vectors and for outputting said motion vec- 
tor (61 ) together with a memory instruction flag 
(61 ') indicative of either one of the first and sec- 
ond comparison image information memory 
means corresponding to said selected motion 
vector. 

An image signal recording apparatus having func- 
tions for 

dividing image data of one picture plane into a 
plurality of blocks, 

forming an evaluation value table based on a 
difference between the present image data and 
the image data of the block which is one or a 



12 



BNSDOCID: <EP 076277 4A2_I_> 



p 



EP 0 762 774 A2 



24 



few frames before every pixel with respect to 
each block, 

detecting coordinates of a minimum value in 
said evaluation value table, 

detecting a motion vector on the basis of said 5 
coordinates, and 

compressing the image data by using said mo- 
tion vector, comprising: 

means (12) for detecting a camera shake mo- 
tion vector from said evaluation value table and 10 
said motion vector; 

means (7) for performing a motion compensa- 
tion by said motion vector; and 
means (5, 8) for executing a compressing proc- 
ess by said input image data and the image da- '5 
ta which was motion compensated, 
and wherein said camera shake motion vector 
is recorded together with said compressed im- 
age signal. 

20 

11. An image signal reproducing apparatus for 

dividing image data of one picture plane into a 
plurality of blocks, 

detecting a motion vector corresponding to a 25 
position of the block which most coincides from 
the image data which is one or a few frames 
before with respect to each block, 
compressing the image data by using said mo- 
tion vector, 30 
detecting a camera shake motion vector from 
said motion vector, 

performing a motion compensation by said mo- 
tion vector, 

executing a compressing process by input im- 35 
age data and the image data which was motion 
compensated, and 

reproducing a recording medium on which said 
camera shake motion vector has been record- 
ed together with said compressed image signal 40 
and said motion vector, comprising: 
means (63) for separating a reproduction signal 
into said compressed image signal, said motion 
vector, and said camera shake motion vector; 
decoding means (64-66) for decoding said ^5 
compressed image data by using said motion 
vector: and 

means (4) to which a decoded image signal 
from said decoding means is supplied and 
which executes a camera shake correction by so 
said camera shake motion vector. 

1 2. An apparatus according to claim 1 1 , further having: 

first image information memory means (101) to 55 
which said decoded image data to be subjected 
to the camera shake correction is inputted; 
second image information memory means 



(1 02) in which peripheral image data of a frame 
in said decoded image has been accumulated; 
and 

selecting means (103) for selecting one of the 
image data outputted from said first and second 
image information memory means in accord- 
ance with said camera shake motion vector. 

13. An image signal reproducing apparatus for repro- 
ducing the image data recorded by the image signal 
processing apparatus according to claim 5, com- 
prising: 

means (63) for separating a reproduction signal 
into said compressed image signal, said motion 
vector, said camera shake motion vector, and 
said image memory instruction flag; 
decoding means (64-66) for decoding said 
compressed image data by using said motion 
vector; and 

means (4) to which the decoded image signal 
from said decoding means is supplied and 
which executes the camera shake correction by 
said camera shake motion vector, 
wherein said decoding means is constructed by 
first image information memory means (101 ) in 
which the image data of one or a few frames 
before has been stored, 

second image information memory means 
(102) in which background image data existing 
in a plurality of frames has been accumulated, 
and 

means (1 03) for selecting either one of said first 
and second image information memory means 
in accordance with said image memory instruc- 
tion flag and executing the motion vector cor- 
rection. 

14. An apparatus according to claim 13, wherein said 
camera shake correcting means (4) selects and 
outputs 

the decoded image data to be subjected to the 

camera shake correction and 

the background image data which is obtained 

from said second image information memory 

means 

in accordance with said camera shake motion 
vector. 

15. An image signal reproducing apparatus for repro- 
ducing the image data recorded by the image signal 
recording apparatus according to claim 10, com- 
prising: 

means (63) for separating a reproduction signal 
into said compressed image data and said 
camera shake motion vector; 
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decoding means (64-66) for decoding said 
compressed image data; and 
means (4) to which the decoded image signal 
from said decoding means is supplied and 
which executes the camera shake correction by s 
said camera shake motion vector. 

16. An image signal recording method having functions 
for 

w 

dividing image data of one picture plane into a 
plurality of blocks, 

detecting a motion vector corresponding to a 
position of the block which most coincides from 
the image data of one or a few frames before is 
with respect to each block, and 
compressing the image data by using said mo- 
tion vector, comprising the steps of: 
detecting the motion vector of each block; 
detecting a camera shake motion vector from 20 
the motion vector of each of said blocks; 
correcting an input image signal on the basis of 
the camera shake motion vector: 
obtaining the motion vector in which the motion 
vector of each of said blocks has been correct- 25 
ed by the camera shake motion vector; 
performing a motion compensation by said cor- 
rected motion vector; and 
executing a compressing process by said im- 
age data which was camera shake corrected 30 
and said image data which was motion com- 
pensated. 

17. An image signal recording method having functions 

for 35 

dividing image data of one picture plane into a 
plurality of blocks, 

detecting a motion vector corresponding to a 
position of the block which most coincides from *o 
the image data of one or a few frames before 
with respect to each block, and 
compressing the image data by using said mo- 
tion vector, comprising the steps of: 
detecting the motion vector of each block; *s 
detecting a camera shake motion vector from 
the motion vector of each of said blocks; 
performing a motion compensation by said mo- 
tion vector; and 

executing a compressing process by input im- so 
age data and the motion compensated image 
data, 

wherein said camera shake motion vector is re- 
corded together with said compressed image 
signal. 55 

18. An image signal reproducing method of 



dividing image data of one picture plane into a 
plurality of blocks; 

detecting a motion vector corresponding to a 
position of the block which most coincides from 
the image data of one or a few frames before 
with respect to each block, 
compressing the image data by using said mo- 
tion vector, 

detecting a camera shake motion vector from 
said motion vector, 

performing a motion compensation by said mo- 
tion vector, 

executing a compressing process by input im- 
age data and the motion compensated image 
data, and 

reproducing a recording medium on which said 
camera shake motion vector has been record- 
ed together with said compressed image sig- 
nal, comprising the steps of: 
separating said compressed image signal and 
said camera shake motion vector from a repro- 
duction signal; 

decoding said compressed image data; and 
supplying the decoded image signal from said 
decoding step and executing a camera shake 
correction by said camera shake motion vector. 
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(54) Image signal encoding apparatus 



(57) An interf rame difference Pij and a motion vector 
Vhv are supplied to a camera shake detecting circuit 
(12) from a motion vector detecting circuit (6). The cam- 
era shake detecting circuit (12) integrates the interf rame 
difference and forms an evaluation value ZPhv. When 
none of the blocks of one picture plane satisfy (Vhv < 
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TH2, the camera shake detecting circuit (12) detects a 



camera shake. The motion vectors of the blocks are dis- 
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vector of each macroblock. A camera shake motion vec- 
tor of the whole picture plane is formed from the motion 
vector of each macroblock. A correction signal is formed 
from the detected camera shake motion vector. A cam- 
era shake correcting process is executed using the cor- 
rection signal. 
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